The present invention relates to optimization apparatus for photographic image data with a data input device an image data optimization device and an output device. It further relates to a photographic copier or printer with an optimization apparatus and a process corresponding to the optimization apparatus and a program for carrying out the process.
The term xe2x80x9cphotographyxe2x80x9d as used herein refers to the (especially permanent) capturing of images produced by electromagnetic radiation (especially light) with means suited therefor (for example photographic apparatus with film, digital photographic camera with CCD chip, film camera, video camera and so on).
The field of the present invention relates to the processing of photographic image information, which represents the photographically produced image. The photographic image information, for example, is conventionally captured or stored on a film. The photographic image information so stored can then be converted into digital photographic image data, for example by way of a scanner. But, the photographic image information can also be only digitally captured right from the start, for example by way of a digital camera. It can then be electronically stored, for example, (CD-ROM, DVD) and can, for example, be transmitted through a network (for example LAN or Internet).
The present invention relates to the processing of photographic image data (in the following also called first image data) which represent images produced by photography. The photographic image data are processed with a data processor in such a way that they are suited for picture production. The picture production can thereby be carried out, for example, on a monitor, on (light sensitive) photographic paper by suitable exposure or by way of a printer or by way of another photographic copier apparatus. However, the quality of the representation of the photographic image data is often unsatisfactory whether done conventionally, for example, on photographic paper or as slide or by way of a monitor (for example LCD screen or TV screen) or printer.
A significant cause for the unsatisfactory quality resides in the capturing and storage of the images or the photographic image information. The causes herefor reside, for example, in the optical capturing of the image by the camera, namely, for example, in the quality of the lens system, in the operation of the camera by the user, namely, for example, incorrect exposures, in the quality of the image capturing means, namely the film or the CCD, in the illumination of the photographic object, and so on. The less than optimal conditions during the image capturing and storage of the photographic image information result in so-called color shifts, which means a fixed, preselected color shade is differently captured and stored depending on the color density or brightness. Thus, a color shade shift results relative to the actual color shade to be represented. In other words, the captured and stored image information does not correspond with respect to the color values to the color values of the photographed object. If the photographic image information is further processed prior to producing a picture, for example, by scanning a film with a scanner, this can result in further changes of the color values.
The photographic image data are conventionally obtained from photographic image information by way of color filters. This is the case for example, with a scanner but also with the digital photographic camera. The data so obtained describe thereby the intensity of different colors of an image point. Conventionally, the representation is done with so-called RGB image data in the so-called RGB color space, or by RGB stands for red, green and blue. Respectively one coordinate of the color space is thereby reserved for one color, i.e. for red, green or blue.
It is an object of the invention to optimize photographic image data, such as, for example, the RGB image data. This means that errors made during the photographic capturing and/or storage and/or processing of the photographic image information (especially color value changes) should be reversed, if possible.
This object is achieved by the features of the independent claims. Preferred embodiments are apparent from the dependent claims.
Preferably an optimization apparatus is provided in accordance with invention which optimizes (digital) photographic image data. The optimization apparatus carries out an optimization transformation. The optimization transformation corresponds to a combination of a first transformation which brings the first image data from the first color space into the second color space, the above described correction transformation and a second transformation, which transforms the first image data corrected in the second color space into a third color space, which is suited for the picture production. The third color space may correspond to the first or second color space or may be any other color space. The third color space may also be different from the second color space. The process in accordance with invention also carries out such an optimization transformation together with the improvements described below. The program in accordance with invention is able to run especially on the computer or a workstation and carries out the process steps in accordance with invention.
The optimization apparatus includes a (digital) data input device for the input of first image data which are the photographic image data. The data input device can be, for example, a data interface, a modem or a scanner. If it is a scanner, the first (photographic) image data are, for example, captured by scanning a film which represents the photographic image information to be processed. Alternatively, the first image data can also be obtained, for example, through a network. These first image data are present in a first color space. Typically, this is a color space wherein the coordinates respectively describe one color i.e., for example, the RGB color space. The first image data represent an image or several images which have been photographically captured. Subsequent to the data input device is the optimization device in accordance with invention in which the first image data are optimized. Subsequent to the optimization device is then the data output device in order to output the optimized first image data which now optimally represent the image or the images. The data output device can also be a data interface for a modem. The data interface is, for example, connected with a computer or a photographic copier or with a network. However, the data output device can also be a picture producing device, which on a medium produces a picture based on the image data. For example, it can be a printer which based on image data prints the optimized picture. The printing can be carried out, for example, on normal paper or light sensitive paper by suited printers or other photographic copiers. The optimized first image data thereby describe image information which is processed by the data output device. This processing can include especially an adaptation and thereby change of the optimized first image data with regard to the output apparatus. Depending on whether the image data are intended for a monitor or printer of a specific type and depending on the dynamic range of the output medium (photographic paper, normal paper, monitor) the optimized image data can be manipulated and adapted to the dynamic range in order to achieve the best compromise with regard to the output medium. For example, the optimized image data can be digitally overlaid with a mask which brightens or darkens certain image regions in order to carry out an adaptation to the output medium and to the subjective human observation capabilities.
For optimization of the first image data, the latter must be corrected. The inventors have found that it is of significant advantage for the execution of such a correction when it is described by a transformation which corresponds to a correction which is carried out in another color space suited for the correction. The first image data to be optimized are present in a certain color space before the optimization which is in the following referred to as xe2x80x9cfirst color spacexe2x80x9d. The color values in the first color space described by the individual first image data are to be changed by the optimization. This change is preferably carried out by an optimizing transformation. In order that the optimizing transformation uses the advantage of a correction in a suitable color space, the optimizing transformation preferably corresponds to a combination of transforms, namely a so-called first transform, the mentioned correction transform and a so-called second transform, which have the properties described in the following.
The first transform transforms the first image data to be optimized from the first color space into the other, second color space suited for the correction. When the first image data are then found in the second color space, which is different from the first color space, a correction transform is carried out therein, which corrects the first image data within the second color space. When the first image data are corrected, they are transformed by a second transform into a third color space, which is especially different from the second color space and suited for the image reproduction (for example, photographic copier, monitor). In particular, the third color space is again the same color space as the first color space or a standard color space, such as, for example, sRGB or Lab. However, the transformation into the standard color space can also be carried out in a separate step, as described further below. If the first image data are, for example, in the RGB color space, and if the printout is to be carried out by way of a color laser printer including cyan, magenta, yellow and black toner, the transformation can be carried out in a corresponding CMYK color space (xe2x80x9cCxe2x80x9d for cyan, xe2x80x9cMxe2x80x9d for magenta, xe2x80x9cYxe2x80x9d for yellow and xe2x80x9cKxe2x80x9d for black). If the print production is carried out, for example, by exposure of photographic paper to differently colored lasers of three different wavelengths, the transformation can be carried out in a third color space the coordinates of which describe the wavelengths of the 3 lasers.
By transformation of the image data to be processed for the print production into a color space suited for the correction, optimal correction results can be achieved with little processing cost.
The correction transformation in the second color space corresponds preferably to a transform which includes a rotation and/or shear in this second color space. In particular, errors during the photographic capturing, especially errors in the color values (for example, color shift errors), can be corrected with this rotation. Preferably, the second color space is selected such that at least one coordinate describes the color density of the color. This has proven especially advantageous for carrying out a correction, especially when a color density transformation, in particular a rotation and/or shear and/or shift of the coordinate origin is used for the correction or if the color density transformation supports the correction. The color density transformation, especially rotation or shear, is preferably structured such that image data which represent a gray value or gray shade are after the color density transform closer to and/or at a smaller distance from the axis which describes the color density in the second color space (see also FIG. 5 of EP 0 586 772 A1 and description). Preferably a determination of the correction transformation is not carried out based on the first image data to be corrected, in particular because of the large data volume of these first image data. Preferably, the determination of the correction transform and especially the rotation is carried out based on the analysis of second image data. Although these second image data can be directly connected with the first image data (for example, statistically correlated therewith), they cannot be transformed into the first optimized image data because of their information content which is different from the one of the first image data. This is in particular due to the reduced local resolution, as will be described further below.
Preferably, not only a simple offset correction is carried out in the correction transform but a multidimensional correction or transform, and in particular the above-mentioned rotation or shear is carried out.
As mentioned, the correction transformation is preferably based on the analysis of second image data which preferably are also present in the second color space. Based on this analysis, which is in particular a statistical analysis of the second image data, the correction, in particular a multidimensional transformation or rotation, is then carried out in the second color space.
The second image data preferably have a significantly smaller data size than the first image data to be corrected in order to so minimize the processing cost for the analysis and the correction. In particular, the second image data have a significantly smaller local resolution than the first image data. Preferably, the second image data represent the image or images, which are also represented by the first image data. They at least represent a part of the image or images. Thus, a (statistical) correlation preferably exists between the first image data and the second image data. The second image data can be in particular image data which have proven a suitable basis for an analysis and correction. They can be continuously improved throughout the period of operation of the optimization apparatus. In particular, different second image data can be used depending on the type of the first image data to be processed, which means the film type used, the camera type used, and so on.
If the second image data are at least partly obtained from the same photographic image information as the first image data, this can take place independent of the extraction of the first image data or the second image data can be derived from the first image data. An independent extraction of the second image data is for example the scanning of a picture (for example film), which is the basis of the first image data, by a separate scanner, which in particular can have a higher spectral (but lower local) resolution than a scanner which is used for the extraction of the first image data.
If the second image data are obtained from the first, the first image data are therefor transformed in the second color space. Furthermore, a reduction of the local resolution preferably takes place in order to minimize the data volume for the analysis, based on the second image data.
The data volume and/or the local resolution of the second image data is preferably at least one order of magnitude below the data volume of the local resolution of the first image data, which are corrected based on the second image data, more preferred is a data volume or local resolution which is at least two orders of magnitude smaller and especially preferred is a data volume or local resolution which is at least three orders of magnitude smaller. A local resolution of less than 10,000 image points per image for the second image data and especially of the magnitude of about 1,000 image points has proven sufficient.
Processes are used for the analysis and correction, which are known from EP 0 586 773 A1 or U.S. Pat. No. 5,365,353, which are both incorporated herein by reference. However, no optimization takes place therein of the data to be processed for the image reproduction, but a determination of control data for the control of an exposure unit based on image data. A transformation of the control data from the color space in which the correction is carried out to another color space suited for the image reproduction does not take place. The actual image reduction is carried out by exposing a film onto a photographic paper. Especially, in contrast to the present invention, the image information to be reproduced (and stored on the film) is not transformed into (digital) image data or input as digital image data which are then subjected to the optimization transformation in accordance with invention.
Furthermore, in contrast to the processes described in the above-mentioned references, those data which are corrected are preferably not also used for the basis of the analysis. Preferably, a special data set, namely the already mentioned second image data, is produced which is then used as a basis for the analysis, whereby the result of the analysis is then not used for the correction of the second image data, but for the correction of the first image data which are present in particular in high resolution.
As is known from EP 0 586 773 A1 or U.S. Pat. No. 5,365,353, corrections can be carried out which are based on a statistical analysis of one set of images or on a statistical analysis of a single image. The set of images is thereby selected such that a statistical correlation between the images results, for example on the basis of the same film being used or the same camera being used. The second image data are therefore based in this case both on the image presently to be transformed as well as on a set of images in which the image presently to be transformed is included.
Through the correlation between the first and second image data all first image data which represent a certain image are therefore associated with a certain correction transformation.
Several correction transformations can also be used for the correction of first image data which represent a single image. This can be an advantage especially when the image has several dominant areas which greatly vary with respect to brightness and/or color shade. In that case, different correction transformations optimally suited for the respective region of the image can be determined and used in the optimization transformation.
As already mentioned above, undesired changes of the color values of the individual image points can be caused during the photographic recording and/or storing of the photographic image information. The data input device, especially a scanner, can also cause further undesired changes, especially when the scanner has a so-called color shift. The variance properties can be exactly measured for each scanner, for example, by way of color tables. However, it is necessary to carry this out for all color shades and color density levels. As a result of these measurements, a so-called profile of the data input device is obtained, especially of the scanner, for example, in matrix form or in the form of a table. Based on this profile, each individual image point of the first image data can then be corrected so that the variance properties of the input device can be equalized or reversed.
Alternatively, the second image data can be obtained from the first image data through a compensation transformation of the reduced first image data in the second color space. This compensation transformation is based preferably on a statistical analysis of reduced first image data, which represent a multitude of images input through the data input device and which reflect the color value variance properties of the data input device. The color value variance properties of the data input device are at least about equalized through the compensation transformation. A Karhunen-Loxc3xa8ve-Transformation can be used, for example, as compensation transformation. If the data input device is a scanner, a multitude of different film types and image types are preferably scanned in order to obtain base data for the analysis.
For data input devices which are constructed as a data interface, and which do not change the image data or image information, it is not necessary to generate a profile for the data input device or to use a compensation transformation. One can forego the profile even with a scanner, but at the expense of the data output quality.
Once first image data have been obtained in the manner mentioned above, which data are free of influences of the data input device, they are then advantageously subjected to the optimization transformation, or the processing operations for clearance of the first image data from these influences are included in the optimization transformation. For example, the first transformation can be implemented by the mentioned compensation transformation. Subsequently, the transformation from the first color space into the second color space takes place as well as the clearance of the first image data from influences of the data input device. Subsequent to the optimization transformation, another transformation into a standard color space, for example, sRGB or Lab, can take place, which is adjusted to the human eye or the human color perception and is independent of the output apparatus. With this transformation into the standard color space, a dynamic range and a gradation of the transformed image data can be achieved which are adapted to an average paper. An alignment of the dynamic range and gradation can subsequently take place. Such a transformation into the standard color space is carried out especially when the input first image data originate from a film. The dynamic range of a photographic paper, for example, is substantially smaller than the dynamic range of a film with which the photographic image information was captured. Especially the gradation of the photographic paper must be taken into consideration. In contrast, a transformation into the standard color space may not be necessary, when the first image data originate from a digital camera. Subsequent to the optional transformation into the standard color space, an adaptation of the optimized and standardized first image data to the output medium can be carried out. The same applies, albeit to a lesser degree, when the data are to be displayed on a monitor. The image data prepared in this way for a specific output can then be transferred to the output device or stored on a storage medium or transferred through a network (for example Internet) to a data receiver.
It is especially preferred to shape the second image data in such a way that they are free of influences from the data input device. Preferably, this applies especially also to the first image data transformed by the first transformation.
The first and second transformation can be determined depending on whether, for example, the second image data are obtained independent from the first image data, or whether they are obtained from the first image data before or after the reduction of the image data and before or after the application of a profile. The different variants are described in the following by way of a digital system and a hybrid system.